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Abstract 


The  purpose  of  this  report  is  to  summarize  the  work  conducted  within  Project  15au  “Combat 
Identification.”  The  report  begins  with  a  review  of  the  project  objectives  and  provides  some 
background  concerning  the  need  for  this  work.  The  project  achievements  and  cost  performance 
are  summarized,  followed  by  descriptions  of  the  major  work  elements  undertaken.  Among  the 
major  outcomes  are  a)  a  computer-based  methodology  for  studying  the  effects  of  environmental 
(e.g.,  types  of  visual  cues)  and  system  (e.g.,  blue-force  tracking)  factors  on  the  speed  and 
accuracy  of  human  combat  identification  judgments,  b)  an  information  accumulation  model  of 
human  combat  identification  decision  making,  and  c)  preliminary  experimentation  to  validate  this 
model  and  provide  insight  into  the  likely  effects  of  blue-force  tracking  and  rifle-mounted  combat 
identification  assist  systems  on  human  decision  making.  A  complete  list  of  lessons  learned  from 
the  project  is  provided.  The  report  ends  with  a  discussion  of  potential  avenues  for  exploitation  of 
the  project  results  and  suggestions  of  future  research  and  development  directions. 


Resume 


Le  present  rapport  vise  a  resumer  le  travail  effectue  dans  le  cadre  du  Projet  15au  «  Identification 
au  combat ».  II  commence  par  un  examen  des  objectifs  du  projet,  ainsi  qu’une  mise  en 
perspective  sur  ce  qui  rend  le  travail  necessaire.  II  y  a  un  resume  des  reussites  du  projet  et  de 
revolution  des  couts,  puis  des  descriptions  des  elements  de  travail  majeurs  entrepris.  Citons 
parmi  les  resultats  principaux  :  a)  une  methodologie  informatique  pour  etudier  les  effets  de 
Tenvironnement  (types  d'indices  visuels,  par  exemple)  et  du  systeme  (blue  force  tracking,  le  suivi 
des  forces  bleues,  par  exemple)  comme  facteurs  de  la  vitesse  et  de  Texactitude  des  jugements 
humains  d'identification  au  combat  humaine;  b)  un  modele  d’accumulation  d’information  pour  la 
prise  de  decision  en  identification  au  combat;  c)  des  experiences  preliminaires  pour  valider  le 
modele  et  donner  une  idee  des  effets  probables  des  systemes  d’aide  (blue  force  tracking  et 
systeme  d’identification  au  combat  monte  sur  fusil)  sur  la  prise  de  decision  humaine.  Est  incluse 
une  lise  complete  des  logons  apprises  grace  au  projet.  Le  rapport  s’acheve  avec  une  discussion 
des  fagons  dont  on  pourrait  exploiter  les  resultats  du  projet  et  avec  la  suggestion  d’ orientations  de 
recherche  et  de  developpement  a  Tavenir. 
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Executive  summary 


Combat  Identification  (15au):  Project  summary  and  closeout 
report 

David  J.  Bryant;  DRDC  Toronto  TR  2009-128;  Defence  R&D  Canada  -  Toronto; 
July  2009. 

Background:  This  project  focused  on  the  human  factors  pertaining  to  Combat  Identification 
(CID)  with  the  goal  of  deriving  a  better  understanding  of  whether  or  not  this  has  the  potential  to 
enhance  situation  awareness  (SA)  and  target  identification.  Human  factors  also  mediate  the 
impact  of  physical  and  operational  factors  on  CID.  Building  a  model  of  human  CID  decision 
making  is  an  essential  step  toward  improving  CID  performance  and  reducing  incidents  of 
fratricide,  through  the  development  of  decision  support  devices,  improved  training,  revised 
doctrine,  and  so  on. 

The  objective  of  this  project  was  to  develop  and  evaluate  a  cognitive  model  of  human 
CID  decision  making  in  the  Land  Force  context.  In  specific  terms,  this  required  a  number  of 
work  elements. 

First,  we  defined  the  CID  task  environment  (functions  and  decision  requirements)  and  developed 
and  tested  algorithms/heuristics  for  multiple  uncertain  cue  integration.  Modeling  was 
accomplished  by  function  flow  and  decision  requirement  analyses  that  documented  the  formal 
procedures  of  CID  and  described  all  decision  tasks  involved. 

Second,  we  developed  and  evaluated  models  of  human  information  aggregation  and  made 
recommendations  for  human  information  aggregation  processes. 

Third,  this  project  investigated  how  humans  use  advice  from  automated  systems  and  identified 
factors  that  affect  the  calibration  of  judgments  and  factors  that  affect  trust  in  advice.  We  also 
determine  how  people  deal  with  information  uncertainty. 

Major  Deliverables:  Major  deliverables  for  this  project  included  a  revised  version  of  the 
Instrumented  Military  Modelling  Engine  for  Research  using  Simulation  and  Virtual 
Environments  (IMMERSIVE)  software  as  a  platform  for  experimentation  on  human  performance, 
and  a  computer-based  methodology  for  studying  the  effects  of  environmental  (e.g.,  types  of  visual 
cues)  and  system  (e.g.,  blue-force  tracking)  factors  on  the  speed  and  accuracy  of  human  combat 
identification  judgments. 

Major  deliverables  also  included  several  reports  characterizing  the  CID  process  in  terms  of  the 
functional  and  decision  requirements  placed  on  decision  makers.  The  literature  relevant  to  the 
CID  process  was  reviewed  in  relation  to  human  information  processing.  Analysis  of  this 
literature  provided  a  description  of  the  CID  process  for  the  individual  soldier  and  identified  key 
issues  that  impact  the  process  of  evaluating  a  contact  of  interest.  A  function  flow  analysis 
identified  the  component  tasks  and  processes  involved  during  CID,  along  with  the  hierarchical 
relationships  (workflow)  between  these  tasks.  Decision  descriptions  included  information 
requirements,  ratings  of  cognitive  workload,  decision  complexity,  decision  criticality,  time 
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requirements,  CID  stage  (Detect,  Classify,  Recognize,  Identify  or  Act),  decision  outcome,  and 
specific  constraints. 

Additional  work  delivered  reports  documenting  models  of  information  aggregation.  By 
categorizing  the  methods  of  information  aggregation  based  on  their  associated  decision  making 
frameworks,  we  were  able  to  identify  several  heuristics,  cognitive  biases,  and  models  as  being 
specifically  relevant  to  the  tasks  of  weighing  and  integrating  information  as  practiced  in  CID. 
Based  on  a  review  of  the  CID  literature,  we  developed  the  Arousal-Excitation-Activation 
Threshold  (AEAT)  model,  which  is  a  generalized  version  of  an  information  aggregation  model 
intended  to  provide  a  framework  for  understanding  the  CID  decision  process. 

To  perform  an  experimental  evaluation  of  the  AEAT  model  of  CID  decision  making  [14],  a 
revised  version  of  the  IMMERSIVE  software  was  developed  to  support  measurement  of  human 
performaance.  With  this,  a  major  outcome  of  the  project  has  been  a  computer-based 
methodology  for  CID  assessment.  Results  of  the  experimental  evaluation  generally  supported  the 
AEAT  model  as  a  potential  framework  in  which  to  further  study  human  CID  decision  making. 

In  another  work  package,  we  studied  factors  that  affect  human  trust  and  reliance  on  the  CID 
systems,  with  the  ultimate  goal  of  helping  them  better  utilize  the  systems  and  reduce  fratricide 
incidents.  The  work  began  with  a  literature  review  pertaining  to  operator  trust  in  automation. 
Three  experiments  investigated  the  role  of  system  reliability  on  users’  trust  of,  and  reliance  on, 
the  system.  Although  users  do  not  necessarily  have  well-calibrated  levels  of  trust  in  support 
systems,  their  use  of  a  system  is  guided  by  their  trust  in  it.  The  results  of  a  third  experiment 
indicated  that  when  the  reliability  information  was  displayed  as  a  degrading  graphical  element 
method,  sensitivity  increased  and  also  that  an  integrated  (rather  than  separated)  display  format 
affords  more  appropriate  reliance  on  the  system.  The  findings  highlight  the  importance  of 
human-machine  interface  (HMI)  design  on  engendering  appropriate  trust  and  reliance  on  the 
automated  decision  aid. 

The  project  also  produced  reports  documenting  experiments  that  examined  how  people  deal  with 
uncertainty  associated  with  visual  and  behavioral  cues  for  target  identity.  Results  indicated  that 
the  accuracy  of  CID  decisions  can  be  affected  by  uncertainty  and  could  potentially  increase  the 
risk  of  fratricide. 

Lessons  Learned:  Among  the  major  outcomes  of  the  project  are:  a)  a  computer-based 
methodology  for  studying  the  effects  of  environmental  (e.g.,  types  of  visual  cues)  and  system 
(e.g.,  blue-force  tracking)  factors  on  the  speed  and  accuracy  of  human  combat  identification 
judgments;  b)  an  information  accumulation  model  of  human  combat  identification  decision 
making;  and  c)  preliminary  experimentation  to  validate  this  model  and  provide  insight  into  the 
likely  effects  of  blue-force  tracking  and  rifle-mounted  CID  systems  on  human  decision  making. 
A  complete  list  of  lessons  learned  from  the  project  is  provided. 

Future  R&D  and  Exploitation  Activities:  This  section  outlines  future  research  directions  and 
potential  exploitation  activities  for  CID  research  consistent  with  Canadian  Forces’  Command, 
Control,  Communications,  Computers,  Intelligence,  Surveillance  and  Reconnaissance  (C4ISR) 
needs.  The  report  discusses  the  use  of  the  project  results  for  enhancing  doctrine,  training,  and 
procedures,  refining  CID  decision  models,  and  development  and  assessment  of  CID  support  tools. 
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Future  research  could  be  directed  to  developing  a  common  framework  for  evaluation  of  CID 
support  technologies  as  well  as  development  of  computer-simulation-based  training  methods. 
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Combat  Identification  (15au):  Project  summary  and  closeout 
report 

David  J.  Bryant;  DRDC  Toronto  TR  2009-128;  R  &  D  pour  la  defense  Canada  - 
Toronto;  Juillet  2009. 

Generality  :  Le  projet  s’est  attache  aux  facteurs  humains  ayant  trait  a  1’ identification  au  combat 
(IDCbt),  le  but  etant  de  mieux  comprendre  si  cela  est  susceptible  d’ameliorer  la  conscience  de  la 
situation  (CS)  et  1’ identification  de  la  cible.  D'autre  part,  les  facteurs  humains  sont  aussi  le 
medium  des  repercussions  de  facteurs  physiques  et  operationnels  sur  lTDCbt.  Etablir  un  modele 
de  la  prise  de  decision  en  IDCbt  humaine  est  une  etape  essentielle  pour  ameliorer  la  performance 
en  IDCbt  et  reduire  les  incidents  fratricides,  grace  a  l’elaboration  de  dispositifs  d’appui  a  la 
decision,  de  formation  amelioree,  de  doctrine  revisee,  etc. 

L’objectif  du  projet  etait  d’elaborer  et  d’evaluer  un  modele  cognitif  de  la  prise  de  decision  en 
IDCbt  humaine  dans  le  contexte  de  la  Force  terrestre.  Plus  specifiquement,  cela  necessitait  divers 
elements  de  travail. 

Premierement,  nous  avons  defini  les  conditions  d’execution  des  taches  d’lDCbt  (exigences  de 
fonctions  et  de  decisions),  puis  nous  avons  elabore  et  teste  des  algorithmes/heuristiques  pour 
l’integration  d’indices  multiples  incertains.  La  modelisation  s’est  effectuee  par  des  analyses  de 
fonctions  et  d’ exigences  de  decision  qui  documentaient  les  procedures  formelles  d'IDCbt  et 
decrivaient  les  taches  de  decision  impliquees. 

Deuxiemement,  nous  avons  elabore  et  evalue  des  modeles  d'agregation  d’information  humaine  et 
fait  des  recommandations  pour  les  processus  d’agregation  d’information  humaine. 

Troisiemement,  le  projet  a  enquete  sur  la  fa9on  dont  les  etres  humains  utilisent  les  conseils  de 
systemes  automatises,  puis  identifie  les  facteurs  qui  affectent  la  calibration  de  jugements 
et  la  confiance  dans  les  conseils.  Nous  determinons  aussi  comment  les  gens  abordent  l’incertitude 
de  1’ information. 

Principaux  resultats  attendus  :  Citons  parmi  les  principaux  resultats  attendus  du  projet 
plusieurs  rapports  caracterisant  le  processus  d’IDCbt  en  terme  d’ exigences  de  fonction  et  de 
decision  imposees  aux  preneurs  de  decision.  Nous  avons  effectue  un  examen  de  la  recherche 
ayant  trait  au  traitement  de  1’ information  humain  dans  le  processus  lTDCbt.  Une  analyse  de  cette 
recherche  a  permis  une  description  du  processus  d’IDCbt  pour  un  soldat  individuel  et  a  identifie 
les  questions  cles  ayant  des  repercussions  sur  1’ identification  d’un  contact  interessant.  Une 
analyse  des  fonctions  a  identifie  les  processus  et  taches  constituants  impliques  dans  lTDCbt,  ainsi 
que  les  relations  hierarchiques  (deroulement  des  operations)  entre  ces  taches.  Les  descriptions  de 
decision  incluaient  les  exigences  en  information,  la  charge  de  travail  cognitif,  la  complexite  de  la 
decision,  son  caractere  essentiel  ou  pas,  les  contraintes  de  temps,  1’ etape  de  lTDCbt  (detecter, 
classer,  reconnaitre,  identifier  ou  agir),  le  resultat  de  la  decision  et  les  contraintes  specifiques. 


vi 


DRDC  Toronto  TR  2009-128 


Un  travail  supplementaire  a  permis  la  production  de  rapports  documentant  des  modeles 
d'agregation  de  l'information.  En  caracterisant  les  methodes  d’agregation  d’information  selon  les 
cadres  de  prise  de  decision  qui  y  sont  associes,  nous  avons  pu  identifier  plusieurs  heuristiques, 
biais  cognitifs  et  modeles  comme  etant  particulierement  lies  aux  taches  devaluation  et 
d’integration  de  l’information  pratiquees  dans  1'lDCbt.  Apres  examen  des  recherches  en  IDCbt, 
nous  avons  elabore  un  modele  de  seuil  eveil-excitation-activation  (SEEA),  version  generalisee 
d'un  modele  d'agregation  de  l'information  visant  a  foumir  un  cadre  pour  la  comprehension  du 
processus  decisionnaire  de  l'IDCbt. 

Pour  effectuer  une  evaluation  experimental  du  modele  SEEA  de  la  prise  de  decision  en  IDCbt 
(Famewo,  Zobarich,  et  Bruyn  Martin,  2008),  a  ete  elaboree  une  version  revisee  du  logiciel 
MMMIRESV  (moteur  de  modelisation  militaire  instrumentalise  pour  la  recherche  a  l'aide 
d'environnements  simules  et  virtuels),  pour  appuyer  la  mesure  du  rendement  humain.  Ceci  etant, 
Pun  des  resultats  principaux  du  projet  a  ete  une  methodologie  informatique  pour  1’ evaluation  de 
l’IDCbt.  Les  resultats  de  revaluation  experimentale  ont  generalement  appuye  le  modele  SEEA 
comme  cadre  potentiel  pour  poursuivre  l'etude  de  la  prise  de  decision  en  IDCbt  humaine. 

Dans  un  autre  ensemble  de  travail,  nous  avons  etudie  les  facteurs  qui  affectent  la  confiance 
humaine  dans  les  systemes  d'IDCbt  et  leur  utilisation,  l’objectif  ultime  etant  d’aider  les  gens  a 
mieux  utiliser  les  systemes  et  de  reduire  les  incidents  fratricides.  Le  travail  a  commence  par  un 
examen  de  la  recherche  portant  sur  la  confiance  accordee  par  un  operateur  a  une  automation. 
Trois  experiences  ont  explore  l'influence  de  la  fiabilite  du  systeme  sur  la  confiance  que  lui 
accorde  l’utilisateur  et  sur  son  utilisation.  Bien  que  les  utilisateurs  n'aient  pas  necessairement  des 
niveaux  de  confiance  bien  calibres  dans  les  systemes  de  soutien,  leur  utilisation  d'un  systeme 
depend  de  la  confiance  qu'ils  lui  accordent.  Une  troisieme  experience  a  montre  que  quand  on 
affiche  la  fiabilite  de  l’information  par  une  methode  d’element  graphique  allant  se  degradant,  la 
sensibilite  augmente,  tandis  qu’un  format  d’affichage  integre  (plutot  que  separe)  permet  une 
utilisation  en  confiance  plus  appropriee  du  systeme.  Les  constatations  soulignent  l’importance  de 
la  conception  de  1’ interface  homme-machine  pour  amener  une  utilisation  en  confiance  plus 
appropriee  de  l’aide  automatisee  a  la  decision. 

D'autre  part,  le  projet  a  produit  des  rapports  documentant  des  experiences  visant  a  determiner 
comment  les  gens  affrontent  une  incertitude  associee  a  des  indices  visuels  et 
comportementaux  quant  a  l’identite  de  la  cible.  Les  resultats  montrent  que  l’exactitude  des 
decisions  d’IDCbt  est  susceptible  d’etre  affectee  par  l’incertitude,  ce  qui  pourrait  augmenter  le 
risque  d'incidents  fratricides. 

Lemons  apprises  :  Citons  parmi  les  resultats  principaux  :  a)  une  methodologie  informatique  pour 
etudier  les  effets  de  l’environnement  (types  d'indices  visuels,  par  exemple)  et  du  systeme  (blue 
force  tracking,  par  exemple)  comme  facteurs  de  la  vitesse  et  de  l’exactitude  des  jugements 
humains  d'identification  au  combat;  b)  un  modele  d’ accumulation  d’information  pour  la  prise  de 
decision  humaine  en  identification  au  combat;  c)  des  experiences  preliminaires  pour  valider  le 
modele  et  donner  une  idee  des  effets  probables  des  systemes  d’aide  (Blue  Force  Tracking  et 
systeme  d’identification  au  combat  monte  sur  fusil)  sur  la  prise  de  decision  humaine.  Est  incluse 
une  lise  complete  des  lefons  apprises  grace  au  projet. 

Activites  a  venir  de  R&D  et  d’exploitation  :  La  section  esquisse  les  directions  de  recherche 
future  et  les  activites  d’exploitation  potentielles  pour  une  recherche  en  IDCbt  correspondant  aux 
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besoins  C4ISR  des  Forces  canadiennes.  Le  rapport  discute  Futilisation  des  resultats  du  projet 
pour  ameliorer  la  doctrine,  la  formation  et  les  procedures,  pour  affiner  les  modeles  de  prise  de 
decision  en  IDCbt  et  pour  elaborer  et  evaluer  des  outils  de  soutien  a  1'IDCbt.  La  recherche  a  venir 
devrait  viser  l’etablissement  d’un  cadre  commun  pour  revaluation  des  technologies  de  soutien  a 
1’IDCbt  et  pour  l’elaboration  de  methodes  de  formation  reposant  sur  la  simulation  informatique. 
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Introduction 


Project  15au,  Combat  Identification,  previously  investigated  ways  to  reduce  fratricide  through 
situation  awareness  in  synthetic  environments,  under  the  management  of  Francois  Bernier  at 
DRDC  Valcartier.  Beginning  in  fiscal  year  (FY)  06/07,  the  project  was  refocused  on  developing 
an  understanding  of  human  combat  identification  (CID)  decision  making,  especially  as  it  affects 
the  viability  of  decision  support  concepts.  This  report  is  a  summary  of  studies  performed  under 
Project  15au,  Combat  Identification,  in  the  fiscal  years  06/07  -  08/09.  David  Bryant  served  as 
Project  Manager  during  this  period. 


Background 

CID  is  the  capability  to  rapidly  and  accurately  identify  friendly,  enemy  and  neutral  forces, 
manage  and  control  the  battlespace,  optimally  employ  weapons  and  forces,  and  minimize  the  risk 
of  fratricide.  As  such,  CID  is  a  complex  cognitive  as  well  as  technological  process.  Although 
improvements  in  sensors  and  development  of  Blue  Force  Tracking  (BFT)  devices  can  enhance  the 
effectiveness  of  CID,  it  is  nevertheless  necessary  to  understand  the  human  decision  making 
processes  involved  in  identification.  The  work  performed  under  this  project  was  aimed  at 
providing  a  comprehensive  and  detailed  description  of  the  CID  domain  as  a  precursor  to 
empirical  study  of  human  CID  decision  making. 

CID  is  sometimes  considered  to  entail  three  elements:  situation  awareness  (SA),  target 
identification,  and  tactics,  techniques,  and  procedures  (TTPs)  (e.g.,  [1]).  SA  refers  to  the 
perception  and  understanding  of  the  operational  environment  needed  to  act  effectively  in  that 
environment.  CID  clearly  requires  SA  as  a  precursor  to  the  classification  of  entities  as  friendly, 
hostile,  or  neutral.  Target  identification  is  the  process  of  making  that  classification  judgment 
based  on  the  characteristics  of  the  entity  in  question  in  relation  to  the  TTPs  that  govern  how  one 
interprets  objects  in  the  operational  environment.  One  can  think  of  SA  as  providing  the  data 
about  objects  in  the  environment  and  TTPs  providing  the  knowledge  needed  to  interpret  that  data. 
Target  identification  is  thus  the  process  by  which  SA  and  TTPs  are  employed  [1]. 

CID  is  a  key  element  of  combat  effectiveness.  It  is  the  process  by  which  enemies  are  identified 
and  targeted  for  destruction.  Nevertheless,  it  is  generally  the  failures  of  CID  that  cause  the  most 
concern  both  operationally  and  nationally.  An  unfortunate  aspect  of  CID  is  the  occurrence  of 
fratricide,  the  inappropriate  engagement  and  potential  wounding  or  killing  of  a  friendly  soldier  or 
unit.  Other  potential  problems  include  neutricide  (incidents  in  which  civilians  and  civilian 
infrastructure  are  accidentally  targeted  or  misidentified  and  deliberately  targeted  [2])  and  injury 
or  death  to  oneself  caused  by  failing  to  identify  an  enemy  contact1  (see  [4]). 

There  is  no  single  cause  for  incidents  of  fratricide  and  neutricide.  The  major  risk  factors  that 
have  been  identified  are  a)  the  loss  of  SA,  and  b)  misidentification  of  the  target  [4]  [5]  [6].  Each 
of  these  factors,  however,  is  itself  a  confluence  of  more  proximal  factors.  These  break  down 
further  into  human,  physical,  and  organizational  factors  [1].  Exploration  of  each  factor  holds  the 
promise  of  generating  solutions  to  the  problem  of  fratricide. 

Human  factors  are  characteristics  or  traits  of  human  beings,  related  to  their  physiology,  cognitive 
capabilities,  and  development  (e.g.,  through  training),  that  can  negatively  affect  CID 
performance.  One  such  factor  is  the  natural  limit  to  information  processing  capacity  exhibited  by 


1  Referred  to  as  a  mistake  akin  to  'suicide'  on  the  battlefield  by  Karsh,  Walrath,  Swoboda  and  Pillalamarri  [3], 
DRDC  Toronto  TR  2009-128 


1 


people,  which  makes  it  difficult  for  soldiers  to  maintain  SA  in  complex  environments  [6].  CID  is 
made  especially  difficult  for  soldiers  in  environments  such  as  Afghanistan  by  the  asymmetric 
nature  of  that  conflict,  characterized  by  a  difficulty  in  knowing  who  one’s  enemy  is,  where  they 
are,  and  how  and  when  they  will  attack.  Human  beings  are  also  subject  to  stress  and  other 
emotions  that  can  impair  performance,  leading  to  misidentifications,  lack  of  fire 
discipline,  etc.  Training  and  education  can  be  positive  factors  but  poor  training  can  impair  both 
SA  and  identification  [1]. 

Physical  factors  include  both  environmental  conditions  and  the  state  of  equipment,  especially 
sensors.  Environmental  conditions  that  reduce  visibility  or  hinder  the  functioning  of  sensors  are 
key  factors  in  many  fratricide  incidents  [7].  Equipment  failures  can  also  make  CID  more  difficult 
and  error-prone.  Increasingly,  operational  zones  will  feature  the  presence  of  similar  or  even 
identical  equipment  being  used  by  friendly,  neutral,  and  enemy  forces  and  this  can  cause 
tremendous  confusion  in  the  identification  process  [6]. 

Operational  factors  pertain  to  the  unique  geographical,  cultural,  and  historic  features  of  the 
operational  setting,  as  well  as  the  organizational  structure  in  which  soldiers  function.  Operating 
afield  in  unfamiliar  nations  often  leaves  soldiers  with  limited  knowledge  of  the  kinds  of 
information  needed  to  distinguish  neutral  from  potentially  hostile  factions.  It  often  is  the  case 
that  such  knowledge  is  difficult  and  time-consuming  to  acquire.  Constraints  imposed  from  higher 
command  in  the  form  of  Standard  Operating  Procedures  (SOPs)  and  Rules  of  Engagement  (ROE) 
can  further  hinder  the  CID  process.  Failures  of  command  and  control  (C2)  and  communication 
frequently  contribute  to  fratricide  and  neutricide  incidents  [7].  All  of  these  issues  are  exacerbated 
in  high-tempo  operations  that  decrease  margins  of  error  [4]. 

Current  project 

This  project  focused  on  the  human  factors  pertaining  to  CID.  There  are  many  ways  in  which 
human  cognition  is  central  to  CID  and  so  deriving  a  better  understanding  of  this  has  the  potential 
to  enhance  SA  and  target  identification.  Human  factors  also  mediate  the  impact  of  physical  and 
operational  factors  on  CID.  Building  a  model  of  human  CID  decision  making  is  an  essential  step 
toward  improving  CID  performance  and  reducing  incidents  of  fratricide,  through  the  development 
of  decision  support  devices,  improved  training,  revised  doctrine,  and  so  on. 


Objectives 

The  objective  of  this  project  was  to  develop  and  evaluate  a  cognitive  model  of  human 
CID  decision  making  in  the  Land  Force  context.  In  specific  terms,  this  required  a  number  of 
work  elements. 

First,  we  defined  the  CID  task  environment  (functions  and  decision  requirements)  and  developed 
and  tested  algorithms/heuristics  for  multiple  uncertain  cue  integration.  This,  in  addition  to  a 
thorough  review  of  the  scientific  and  military  literatures,  provided  a  comprehensive 
description  of  CID  decision  making.  Modeling  was  accomplished  by  function  flow  and  decision 
requirement  analyses  that  will  document  the  formal  procedures  of  CID  and  describe  all  decision 
tasks  involved. 


Second,  we  developed  and  evaluated  models  of  human  information  aggregation  and  made 
recommendations  for  human  information  aggregation  processes. 

Third,  this  project  investigated  how  humans  use  advice  from  humans  and  automated  systems  and 
identify  factors  that  affect  the  calibration  of  judgments  and  factors  that  affect  trust  in 
advice.  We  also  determine  how  people  deal  with  information  uncertainty.  The  results  of  these 
studies  supported  the  development  of  procedures,  training,  and  technology  to  enhance  CID 
decision  making. 

This  work  will  provide  a  framework  in  which  to  understand  how  humans  make  combat  ID 
decisions,  which  will  be  useful  in  identifying  what  kinds  of  support  are  needed,  how  decision 
support  will  be  used,  and  what  impact  various  system  parameters  have  on  human  performance. 


Personnel/Contractors 

DRDC  Toronto 

Dr.  David  J.  Bryant  -  Project  Manager 
Dr.  David  J.  Smith  -  Scientist 
Dr.  Justin  Hollands  -  Scientist 
Elaine  Maceda  -  Research  Assistant 
Matthew  Lamb  -  Research  Assistant 

Sabrina  Kanani  -  University  of  Waterloo  co-op  student,  research  assistant 

HumanSystems  Inc. 

R.  Boothby 
L.  Bruyn  Martin 
T.  Lamoureux 
J.  Famewo 
R.  Zobarich 
P.  Vilhena 
Dr.  M.  Matthews 

University  of  Toronto 

Dr.  G.  A.  Jamieson  -  Project  Lead 
L.  Wang 
H.  F.  Neyedli 


DRDC  Toronto  TR  2009-128 


3 


Achievement  of  the  project  objectives 


Table  1  lists  the  major  milestones  for  this  project.  Milestones  1  through  7  were  completed  prior 
to  fiscal  year  2005/2006,  when  Francois  Bernier  was  project  manager.  They  are  listed  here  for 
the  sake  of  completeness  but  will  not  be  discussed  as  this  report  deal  solely  with  work  performed 
at  DRDC  Toronto. 


Table  1.  Milestone  Completion  Report 

No. 

Milestones 

Planned 

Completion 

Date 

Actual 

Completion 

Date 

Status 

1 

Definition  of  initial  vignette  (scenarios)  set 

1 -Dec-04 

30-Jun-05 

Completed 

2 

Analysis  and  review  of  synthetic  environment  frameworks 
(JCATS,  STRIVE,  OneSAF) 

1 -Dec-05 

1  -Dec-05 

Completed 

3 

Analysis  and  review  of  agent  and  multiagent  systems 

1  -Dec-05 

1 -Dec-05 

Completed 

4 

Identification  of  knowledge  engineering  of  LAV  III 
command  tasks 

1 -Feb-06 

31 -Aug-06 

Completed 

5 

Interim  report  on  virtual  and  constructive  simulation 
objectives  and  progress 

1  -Oct-05 

31 -Mar-07 

Completed 

6 

Report  on  virtual  simulation  for  fratricide  study 

31-Dec-06 

31 -Mar-07 

Partial 

7 

Report  on  constructive  simulation  for  fratricide  study 

31 -Mar-07 

31 -Oct-07 

Completed 

8 

Literature  review  on  probabilistic  cue  integration 

31 -Mar-07 

31 -Aug-07 

Completed 

9 

Report  documenting  functional  analysis 

31 -Mar-07 

31 -Mar-07 

Completed 

10 

Experiment  plan 

31 -Mar-07 

31 -Mar-07 

Completed 

11 

Literature  review  on  information  aggregation  methods 

31 -Mar-07 

31 -Mar-07 

Completed 

12 

Human  use  of  advice  experiment  results 

31 -Mar-07 

31 -Mar-07 

Completed 

13 

Experiment  plan  for  evaluation  of  models  of  information 
aggregation 

31 -Mar-08 

31 -Mar-08 

Completed 

14 

Experimental  methods,  materials,  and  scenarios  for  all 
experiments 

31 -Mar-08 

31 -Mar-08 

Completed 

15 

Report  on  experiments  (results  of  evaluation  of  models  of 
information  aggregation) 

31 -Mar-08 

31 -Mar-08 

Completed 

16 

Human  CID  task  performance 

31 -Mar-08 

31 -Mar-08 

Completed 

17 

Field  evaluation  of  cognitive  models  of  CID  decision 
making 

31 -Mar-09 

31 -Mar-09 

Terminated 

All  major  milestones  performed  at  DRDC  Toronto  (8-17)  were  completed,  with  the  exception  of 
milestone  17.  This  milestone  corresponds  to  a  field  evaluation  that  was  to  have  been  performed 
to  evaluate  models  of  CID  decision  making  under  actual  operational  conditions.  This  work 
element,  however,  was  terminated  due  to  the  inability  of  the  research  team  to  locate  a  field 
exercise  that  would  afford  sufficient  experimental  control  to  perform  a  scientifically  valid 
evaluation.  Most  field  exercises  are  conducted  to  meet  certain  specific  training  objectives  and  so 
organizers  are  very  reluctant  to  alter  their  exercise  plan  in  ways  that  are  necessary  for  an 
experimental  comparison  of  conditions,  as  such  alterations  would  have  significant  impact  on  the 
types  of  activities  in  which  participants  would  be  engaged,  as  well  as  the  pacing  of  the  exercise  as 


a  whole.  The  project  manager  explored  the  potential  for  data  collection  at  Exercise  Bold  Quest 
Plus  (Ex  BQ+),  conducted^  at  Eglin  Air  Force  Base  (AFB)  in  Florida,  July  11-25,  2008. 
Unfortunately,  Ex  BQ+  did  not  allow  for  the  direct  measurement  of  relevant  human  performance. 
In  addition,  key  decision  makers  in  the  CID  process,  the  Forward  Air  Controllers,  participated  as 
part  of  the  team  running  the  exercise  rather  than  players  in  that  exercise  and  so  contributed  no 
data  whatsoever. 

All  deliverables  from  these  milestones  are  listed  in  Annex  A. 
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Schedule  and  cost  performance  summary 


Table  2  presents  a  summary  of  funding  and  spending  for  Project  15au  in  the  FY  06/07  through 
08/09. 


Table  2.  Budgeted  and  Actual  Spending  by  Fiscal  Year 

Fiscal  Year  (FY) 

06/07 

07/08 

08/09 

Budgeted  Allocation 

$160K 

$200K 

$100K 

Actual  Amount  Spent 

$124K 

$196K 

$65K 

Funding  was  under-spent  in  FY  06/07  because  $30  was  transferred  to  WBE  15au02,  delivered  by 
DRDC  Valcartier. 

Funding  was  under-spent  in  FY  08/09  because  no  suitable  opportunity  was  available  to  conduct 
an  experimental  evaluation  in  a  field  exercise  setting.  The  project  manager  explored  the  potential 
for  data  collection  at  Ex  BQ+.  As  the  bulk  of  the  funding  that  year  was  intended  for  such  a  field 
exercise,  it  was  decided  to  not  attempt  to  spend  the  funding  on  an  exercise  that  did  not  provide 
adequate  opportunities  to  collect  human  decision  making  data.  Some  funds  from  this  FY  were 
used  to  support  research  on  deployable  day/night  goggle  simulation  for  another  DRDC  project. 


Summary  of  achievements 


Analysis  of  CID  issues 

Bruyn  Martin,  L.,  Famewo,  J.,  Zobarich,  R.,  &  Lamoureux,  T.  (2007).  Function 
Flow  Analysis  for  the  Combat  Identification  Process.  DRDC  Toronto  Contractor 
Report  (CR  2007-130). 

Vilhena,  P.G.S.,  Zobarich,  R.M.  &  Lamoureux,  T.M.  (2007).  CID  Literature 
Review.  DRDC  Toronto  Contractor  Report  (CR  2007-131). 

Famewo,  J.  J.,  Bruyn  Martin,  L.  E.,  Zobarich,  R.  M.,  &  Lamoureux,  T.  (2007). 
Decision  Requirements  Analysis  for  the  Combat  Identification  Process.  DRDC 
Toronto  Contractor  Report  (CR  2007-132). 

Famewo,  J.  J.,  Bruyn  Martin,  L.  E.,  Zobarich,  R.  M.,  Vilhena,  P.G.S.,  & 
Lamoureux.  T.  M.  (2007).  Combat  Identification:  A  Summary  of  the  Literature, 
Function  Flow  Analysis  and  Decision  Requirements  Analysis.  DRDC  Toronto 
Contractor  Report  (CR  2007-123). 

The  objective  of  work  done  in  this  area  was  to  characterize  the  CID  process  in  terms  of  the 
functional  and  decision  requirements  placed  on  decision  makers.  The  first  step  was  to  conduct  a 
review  of  the  scientific  and  military  literatures.  The  CID  literature  review  [8]  investigated  CID 
from  the  perspective  of  the  dismounted  infantry  and  the  Light  Armoured  Vehicle 
perspective.  The  literature  relevant  to  the  CID  process  was  reviewed  in  relation  to  human 
information  processing.  Analysis  of  this  literature  provided  a  description  of  the  CID 
process  for  the  individual  soldier  and  identified  key  issues  that  impact  the  process  of  evaluating  a 
contact  of  interest. 

The  review  indicated  the  importance  of  several  concepts  to  effective  CID  and  the  reduction  of 
fratricide  incidents:  Situation  Awareness  (SA),  Target  Identification  (TI),  Tactics,  Techniques  and 
Procedures  (TTPs),  and  operational  effectiveness.  Good  SA  will  necessarily  improve  CID 
performance,  because  the  observer  will  have  less  unknown  information  in  the  environment.  The 
implications  for  the  soldier  performing  the  CID  task  are  that  s/he  must  take  more  time  and  search 
for  more  information  in  order  to  complete  TI.  TTPs  affect  the  likelihood  of  a  soldier  coming  into 
contact  with  a  potential  threat,  how  the  soldier  searches  for  that  threat,  how  the  soldier 
aggregates  information  to  arrive  at  a  decision,  and  what  further  action  the  soldier  will 
take  (e.g.  the  ROE  for  an  operation). 

The  review  indicated  that  CID  is  not  a  simple  stimulus-response  task,  but  involves  the 
aggregation  of  information.  The  authors  put  forward  an  information  aggregation  model  of  human 
CID  decision  making  in  which  an  individual’s  activation  threshold  can  be  considered  to  be  the 
comfort  the  soldier  has  with  the  situation,  and  excitation  can  be  considered  the  data  points  the 
soldier  has  about  the  situation.  Both  the  consideration  (value)  of  the  stimulus  and  the  level  of  the 
activation  threshold  are  subjective  to  the  individual  soldier.  The  report  discusses  the  factors 
influencing  the  level  of  the  threshold  and  the  stimuli  that  contribute  to  activation,  ranging  from 
those  that  change  the  level  of  the  activation  threshold,  to  those  that  increase  or  decrease  the  level 
of  excitation. 
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A  function  flow  analysis  was  conducted  to  identify  the  component  tasks  and  processes  involved 
during  CID,  along  with  the  hierarchical  relationships  (workflow)  between  these  tasks.  By 
decomposing  CID  into  its  component  parts  in  a  function  flow  diagram,  we  were  able  to 
evaluate  the  complexity  of  the  CID  task  and  acquire  insight  into  the  activities  and  requirements  of 
the  mounted  and  dismounted  soldier.  A  report  [9]  describes  the  method  used  to  develop  the 
function  flow  diagrams  in  which  the  CID  process  was  identified  and  collected  from  a  review  of 
the  literature,  results  from  nine  multi-player  experiments,  and  interviews  with  Subject  Matter 
Experts  (SMEs). 

Based  on  this  analysis,  we  identified  four  high-level  functions  performed  by  individual  mounted 
or  dismounted  soldiers  during  the  CID  process: 

•  Function  1 :  Prepare  for  the  mission; 

•  Function  2:  Perform  mounted/dismounted  functions; 

•  Function  3:  Take  action;  and 

•  Function  4:  Evaluate  action  taken. 

Each  main  function  was  further  decomposed  to  progressing  levels  of  detail  concerning  the 
decisions  and  actions  of  the  soldier.  The  function  flow  diagrams  detail  the  component  tasks 
performed  by  the  mounted  or  dismounted  soldier  to  at  least  three  levels  of  decomposition. 

With  reference  to  the  typical  definition  of  CID  (described  in  [8]),  Function  1  reflects  the  building 
of  baseline  situation  awareness,  Function  2  mirrors  target  identification  and  Function  3  reflects 
the  TTPs.  Function  4  adds  an  important  element  to  the  CID  task  that  allows  the  soldier  to 
calibrate  his/her  SA  thereby  continually  improving  his/her  CID  skills. 

We  conducted  a  decision  requirements  analysis  to  identify  and  describe  the  decisions  associated 
with  CID  as  performed  by  individual  mounted  and  dismounted  soldiers  in  a  Land  Force  context 
[10].  Decision  descriptions  included  information  requirements,  ratings  of  cognitive  workload, 
decision  complexity,  decision  criticality,  time  requirements,  CID  stage  (Detect,  Classify, 
Recognize,  Identify  or  Act),  decision  outcome,  and  specific  constraints.  The  analysis  was  based 
on  the  CID  functions  previously  diagrammed  through  a  function  flow  analysis  and  a  review  of  the 
CID  literature. 

The  analysis  resulted  in  the  identification  of  67  decisions,  which  were  grouped  and  summarized 
to  represent  seven  core  CID  decisions  distributed  across  the  four  functions.  The  analysis  also 
resulted  in  an  in-depth  description  of  the  decisions  involved  in  each  of  the  four  functions,  such 
that  recommendations  were  possible  regarding  the  needs  of  the  CID  decision  maker,  such  as 
experience,  training,  good  visual  resolution,  availability  and  accessibility  of  information  (e.g., 
regarding  enemy,  friendly,  neutral  forces  and  specific  individuals  of  interest),  meta- cognitive 
skills,  and  intuition. 


Models  of  information  aggregation 

Famewo,  J.,  Matthews,  M.,  &  Lamoureux,  T,  (2007).  Models  of  information 
aggregation  pertaining  to  combat  identification:  A  review  of  the  literature.  DRDC 
Toronto  Contractor  Report  (CR  2007-062). 


Bryant,  D.  J.  (2007).  Classifying  Simulated  Air  Threats  with  Fast  and  Frugal 
Heuristics.  Journal  of  Behavioral  Decision  Making,  20,  37-64. 

We  conducted  a  literature  review  to  examine  methods  of  aggregating  information  in  order  to 
develop  models  that  could  be  applied  to  human  CID  decision  making.  The  review  focused  on 
identifying  ways  to  improve  the  quality  of  decisions  through  optimization  of  formal  and  informal 
models  of  information  aggregation.  A  report  documenting  the  results  of  the  review  [11] 
highlighted  major  principles  and  theories  of  human  information  aggregation  and  developed  a 
categorization  scheme  in  which  to  evaluate  the  applicability  of  models  of  information  aggregation 
to  human  decision  making,  especially  in  the  context  of  CID. 

By  categorizing  the  methods  of  information  aggregation  based  on  their  associated  decision 
making  frameworks,  we  were  able  to  identify  several  heuristics,  cognitive  biases,  and  models  as 
being  specifically  relevant  to  the  tasks  of  weighing  and  integrating  information  as  practiced  in 
CID.  Early  analytic  approaches  to  explaining  decision  making  were  based  on  normative 
theories  of  probability  and  logic,  whereas  intuitive  methods  encompass  a  range  of  informal 
heuristics.  The  review  found  that  the  characteristics  of  combat  identification  fit  it  to  the  intuitive 
decision  making  framework.  By  determining  the  methods  of  aggregation  associated  with  this 
framework  we  were  able  to  form  a  list  of  principles,  heuristics  and  biases  that  may  affect  CID 
decision  makers. 

Based  on  a  review  of  the  CID  literature,  we  developed  the  Arousal-Excitation-Activation 
Threshold  (AEAT)  model  [12],  which  is  a  generalized  version  of  an  information  aggregation 
model  intended  to  provide  a  framework  for  understanding  the  CID  decision  process.  The  AEAT 
model  proposes  that  an  individual  uses  discrete  pieces  of  information,  or  cues,  to  assess  a 
potential  target  [8].  In  this  model,  the  decision  of  whether  to  engage  a  target  as  hostile  is  a 
function  of  the  level  of  arousal  of  the  decision  maker,  where  “arousal”  represents  a  hypothetical 
state  of  psychological  readiness  [13].  The  decision  maker’s  arousal  is  in  turn  a  function  of  the 
available  data  that  provides  confirmatory  evidence  that  the  target  is  hostile.  The  relationship 
among  cues,  arousal,  and  the  CID  decision  is  illustrated  in  Figure  1,  which  presents  a  hypothetical 
plot  of  arousal  level  over  time.  The  baseline  level  of  arousal  indicates  the  existing  tendency  to 
classify  a  target  as  hostile,  whereas  the  activation  threshold  is  the  level  of  arousal  necessary  for 
the  decision  maker  to  assign  the  hostile  designation  to  the  target.  The  arousal  level  fluctuates 
over  time  as  data  are  perceived  and  evaluated.  Data  consistent  with  the  target  being  hostile  raise 
the  arousal  level,  whereas  data  inconsistent  with  it  being  hostile  decrease  the  arousal  level. 

The  soldier’s  activation  threshold  is  the  level  of  excitation  that  must  be  exceeded  for  the  soldier  to 
exercise  his/her  ROEs  (therefore  deciding  that  a  contact  poses  a  threat).  Individual  factors,  such 
as  the  experience  and  training  received  by  the  soldier  (e.g.,  previous  experience  with  a  particular 
location;  beliefs  about  one’s  degree  of  accuracy  on  past  decisions)  and  psychophysiological 
factors  (e.g.,  fatigue,  fear)  affect  the  level  of  the  activation  threshold,  making  it  either  easier  or 
more  difficult  to  reach  the  point  where  one  views  the  situation  as  requiring  an  action. 
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Figure  1.  Information  Aggregation  Over  Time  According  to  the 
Excitation-Activation  Threshold  Model 

The  baseline  arousal  level  is  predicted  to  be  affected  by  psycho-physiological  factors  and 
concepts  such  as  trust  and  confidence,  whereas  the  activation  threshold  is  set  with  respect  to  the 
individuals’  subjective  interpretation  of  tactics,  techniques  and  procedures,  and  ROEs.  Such 
interpretations  could  be  influenced  by  factors  like  stress  and  fatigue  leading  to  variability 
across  individuals. 

Cues  come  from  perceptual  input  from  the  environment  and  outputs  of  artificial  sensors  and  other 
technologies.  The  interpretation  of  cues  depends  on  the  individual’s  knowledge,  derived  from  all 
sources  including  training,  intelligence  briefings,  etc.  Different  cues  carry  varying  weights  or 
strengths  and  effects  (i.e.,  size  of  increase  or  decrease  in  excitation)  based  on  the  certainty  the 
decision  maker  perceives  about  the  information,  and  the  meaningfulness  of  the  cue  [10]. 


Evaluation  of  models  of  human  information  aggregation 

Famewo,  J.  J.,  Zobarich,  R.  M.,  &  Bruyn  Martin,  L,  E.  (2008).  Experimental 
Evaluation  of  the  Combat  Identification  Process.  DRDC  Toronto  Contractor 
Report  (CR  2008-116). 

We  performed  an  experimental  evaluation  of  the  AEAT  model  [14].  The  experiment  made  use  of 
the  Instrumented  Military  Modelling  Engine  for  Research  using  Simulation  and  Virtual 
Environments  (IMMERSIVE)  software  developed  at  DRDC  Valcartier  in  an  earlier  phase  of  the 
project.  IMMERSIVE  is  based  on  a  modified  gaming  environment  called  “Unreal  Tournament” 
and  is  a  first-person  perspective  environment  in  which  the  participant  assumes  the  role  of  a 
dismounted  infantry  soldier.  In  the  experiment,  participants  (28  CF  Reservists)  assumed  the  role 
of  a  dismounted  infantry  soldier  in  a  simulated  environment  and  patrolled  an  area  in  which 


soldiers  appeared  and  moved.  Participants’  decided  whether  each  soldier  was  a  friend,  neutral,  or 
enemy  based  on  criteria  provided  and  whether  to  engage  a  soldier  believed  to  be  hostile. 

The  main  variables  manipulated  in  the  experiment  were  the  strength  (i.e.,  combination  of 
salience,  number,  meaning)  of  the  confirmatory  cues  associated  with  a  contact  and  the  arousal 
level  of  the  participant.  Hostile  contacts  were  indicated  by  the  presence  of  specific  characteristics 
(i.e.,  cues)  that  had  been  identified  in  the  previous  function  flow  analysis  [9].  High  and  low 
strength  targets  were  created  by  varying  the  number  and  salience  of  the  cues  associated  with  a 
given  target.  The  subject’s  baseline  arousal  level  was  manipulated  through  instructions  regarding 
the  presence  or  absence  of  a  generalized  threat  in  a  block  of  trials. 

Each  experiment  session  consisted  of  four  blocks  of  12  trials  (i.e.,  encounters  with  a  target).  The 
first  and  last  block  served  to  establish  baseline  performance  levels.  Each  block  of  trials  began 
with  the  participant  reading  a  short  description  (a  “read-in”)  on  the  computer  screen  regarding  the 
set  of  trials  he  or  she  is  about  to  complete.  In  each  trial,  the  participant  maneuvered  through  a 
village  scene  in  the  simulated  environment.  The  participant  was  provided  with  ROE  that 
distinguished  friendly/neutral  from  potentially  hostile  targets  and  governed  when  the  participant 
was  to  engage  hostile  contacts. 

Participants’  performance  was  assessed  in  terms  of  engagement  accuracy  (correctly  engaging  foes 
but  not  friends  or  neutrals),  confidence  in  engagement  decision,  level  of  engagement  force 
employed,  and  the  cues  used  to  form  the  CID  decision  as  determined  through  an  open  question  at 
the  end  of  each  trial. 

Results  indicated  that  participants  were  more  likely  to  report  the  presence  of  hostile  contacts 
when  their  baseline  arousal  was  heightened  and  when  cues  were  stronger.  This  trend  was  only 
partially  valid  when  considering  the  accuracy  of  participants’  reports.  That  is,  participants  were 
more  accurate  about  the  presence  of  a  hostile  contact  (i.e.,  hit)  when  cues  were  strong,  but  were 
not  influenced  significantly  by  heightened  arousal  level.  However,  arousal  level  did  marginally 
affect  the  proportion  of  false  alarms  (i.e.,  identifying  a  friend/neutral  as  hostile)  such  that  more  of 
these  errors  were  made  when  the  arousal  level  was  heightened.  This  finding  suggests  that 
expectations  may  affect  fratricide/neutricide.  Participants’  confidence  was  directly  related  to  cue 
strength  with  stronger  cue  strengths  eliciting  higher  confidence.  Confidence  was  also  somewhat 
affected  by  the  arousal  level,  such  that  confidence  was  lower  for  hits  when  participants  did  not 
expect  the  presence  of  a  hostile  contact  (i.e.,  lower  baseline  arousal). 
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Human  use  of  CID  decision  support  concepts 


Jamieson,  G.  A.,  Neyedi,  H.  F.,  &  Wang,  L.  (2008).  Developing  Human-Machine 
Interfaces  to  Support  Appropriate  Trust  and  Reliance  on  Automated  Combat 
Identification  Systems.  DRDC  Toronto  Contractor  Report  (CR  2008-114). 

Wang,  L.,  Jamieson,  G.  A.,  &  Hollands,  J.  G.  (in  press).  Trust  and  reliance  on  an 
automated  combat  identification  system:  The  role  of  aid  reliability  and  reliability 
disclosure.  Human  Factors. 

Neyedli,  H.,  Hollands,  J.  G.,  &  Jamieson,  G.  A.  (2009,  accepted).  Human  reliance  on 
an  automated  combat  identification  system:  Effects  of  display  format.  Submitted  to 
the  Annual  Meeting  of  the  Human  Factors  and  Ergonomics  Society. 

Wang,  L.,  Jamieson,  G.  A.,  &  Hollands,  J.  G.  (2008).  Selecting  methods  for  the 
analysis  of  reliance  on  automation.  In  Proceedings  of  the  Human  Factors  and 
Ergonomics  Society  -  52nd  Annual  Meeting  (pp.  287-291).  Santa  Monica,  CA: 
Human  Factors  and  Ergonomics  Society. 

Wang,  L.,  Jamieson,  G.  A.,  &  Hollands,  J.  G.  (2008).  Improving  reliability 
awareness  to  support  appropriate  trust  and  reliance  on  individual  combat 
identification  systems.  In  Proceedings  of  the  Human  Factors  and  Ergonomics  Society 
-  52nd  Annual  Meeting  (pp.  292-296).  Santa  Monica,  CA:  Human  Factors  and 
Ergonomics  Society. 

Neyedli,  H.  F.,  Wang,  L.,  Jamieson,  G.  A.,  &  Hollands,  J.  G.  (in  press).  Evaluating 
reliance  on  combat  identification  systems:  The  role  of  reliability  feedback.  In  D.  H. 
Andrews  &  T.  Hull  (Eds.),  Human  factors  issues  in  combat  identification.  Aldershot, 
England:  Ashgate. 

A  variety  of  technical  solutions  have  been  developed  with  the  aim  of  improving  soldiers’  CID 
ability.  The  fact  that  the  contemporary  warfare  often  involves  dismounted  urban  operations 
draws  attention  to  one  of  these  technologies  -  the  individual  combat  ID  system  [15].  The  device 
consists  of  a  gun-mounted  interrogator  and  a  helmet  mounted  transponder.  The  interrogator 
sends  out  a  laser  inquiry,  which  the  transponder  decodes  and  sends  back  a  reply.  The  key 
drawback  of  such  a  system  is  that  it  cannot  positively  identify  a  target  without  a  working 
transponder  device  [16]  [17]  [18].  Therefore,  when  no  transponder  signal  is  received,  the  target 
can  be  hostile,  neutral  or  friendly.  It  appears  soldiers  have  difficulty  relying  on  this  imperfect 
automation  appropriately,  which  may  limit  the  benefit  of  this  technology  [19]. 

Humans  are  prone  to  misuse  or  disuse  imperfect  automation  [20].  Misuse  occurs  when 
individuals  rely  on  automation  inappropriately,  usually  by  over  relying  on  an  imperfect  aid. 
Disuse  occurs  when  participants  under  use  or  reject  the  capabilities  of  automation  [21].  Previous 
studies  have  consistently  demonstrated  that  humans’  trust  in  automation  is  a  major  factor 
that  determines  their  reliance  on  the  automation  [21].  The  goal  of  the  work  conducted 
under  this  portion  of  the  project  was  to  study  factors  that  affect  the  humans’  trust  and  reliance  on 
the  CID  systems,  with  the  ultimate  goal  of  helping  them  better  utilize  the  systems  and  reduce 
fratricide  incidents. 


The  work  [22]  began  with  a  literature  review  pertaining  to  operator  trust  in  automation.  Based  on 
this  review,  two  experiments  were  performed  to  address  gaps  in  this  literature.  Experiment  1  was 
designed  to  determine  whether  providing  system  reliability  information  would  lead  to  appropriate 
trust  and  reliance  on  CID  systems.  Experiment  2  was  designed  to  test  whether  the  differences 
between  simulated  CID  systems  and  real  system  prototypes  would  influence  operator  trust  in,  and 
reliance  on,  the  system  feedback.  Because  previous  empirical  studies  had  not  clearly  defined 
reliance  on  automation  we  developed  a  new  experimental  method  for  this  measurement. 

The  literature  review  revealed  weaknesses  in  previous  measures  of  system  reliance.  Thus,  we 
developed  a  new  measurement  approach  based  on  signal  detection  theory  [23]  [24],  in  which  two 
indicators,  sensitivity  and  response  bias,  characterize  an  observer’s  responses.  Both  measures  are 
derived  from  the  probability  with  which  a  subject  correctly  engages  an  enemy  (hit  rate)  and  the 
probability  with  which  a  subject  incorrectly  engages  a  friend  (false  alarm).  Importantly,  an 
observer’s  optimal  response  bias  can  be  defined  in  any  given  condition  (i.e.,  presence  of  a  given 
type  of  decision  support  system)  with  a  predetermined  signal  rate  and  decision  payoff  structure. 
By  comparing  subjects’  observed  response  bias  to  the  computed  optimal  response  bias,  we  can 
determine  whether  an  observer  over-  or  under-relies  on  the  system.  This  measurement  approach 
is  described  in  detail  by  Wang,  Jamieson,  and  Hollands  [25]. 

The  results  of  Experiment  1  suggested  that  participants’  beliefs  about  the  system  reliability  and 
their  trust  in  the  system  feedback  are  positively  correlated.  The  findings  further  indicated  that 
participants’  trust  in  the  feedback  is  positively  correlated  with  their  reliance  on  the  feedback. 
Thus,  individuals  tended  to  trust  what  they  perceived  to  be  reliably  accurate  systems  and  to  make 
use  of  systems  that  they  trusted.  However,  participants’  beliefs  about  the  system  reliability  and 
their  reliance  on  the  feedback  were  not  highly  correlated,  suggesting  that  trust  acts  as  an 
intermediary  between  belief  and  reliance.  This  indicates  that  an  individual’s  trust  in  a  system  was 
not  well  calibrated  to  the  system’s  true  accuracy.  Thus,  a  person  might  over-rely  on  an  inaccurate 
system  incorrectly  perceived  to  be  accurate  or  under-rely  on  an  accurate  system  misperceived  as 
inaccurate.  Others  results  confirmed  that  participants  had  difficulty  estimating  the  system 
reliability.  However,  informing  participants  of  the  true  reliability  of  the  system  led  to  appropriate 
trust  in,  and  reliance  on,  the  system. 

The  results  of  Experiment  2  suggested  that,  although  the  participants’  reliance  on  the  system 
feedback  was  generally  not  affected  by  activation  mode  and  feedback  form,  their  trust  in  the 
system  feedback  was  influenced  by  them.  These  findings  indicate  that  the  dissimilarity  between 
the  simulated  aids  and  the  real  system  prototypes  can  lead  to  changes  in  humans’  trust  in  the 
system  feedback.  This  change  in  trust  may  then  influence  their  use  of  the  system. 

Displaying  feedback  reliability  information  and  acknowledgement  of  aid  activation  for  a  manual 
system  were  deemed  especially  important  requirements.  Three  methods  of  displaying  these 
requirements  were  suggested:  a  graphical  element  which  degrades  and  dims  as  the  reliability 
decreases  and  two  analogue  proportion  displays,  one  that  was  continuous  and  one  with  discrete 
segments.  The  first  two  of  these  displays  were  examined  in  the  experimental  environment. 

A  third  experiment  was  performed  to  evaluate  different  means  of  presenting  feedback  reliability 
information.  In  particular,  the  experiment  examined  whether  integrating  or  separating  the 
feedback  reliability  and  the  feedback  identification  information  affords  a  performance  advantage, 
and/or  more  appropriate  reliance  on  and  trust  in  the  CID  system.  The  results  of  this  experiment 
indicated  that  when  the  reliability  information  was  displayed  as  a  degrading  graphical  element 
method,  sensitivity  increased  while  an  integrated  (rather  than  separated)  display  format  produced 
more  appropriate  reliance  on  the  system.  The  findings  highlight  the  importance  of  human- 
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machine  interface  (HMI)  design  on  engendering  appropriate  trust  and  reliance  on  the  automated 
decision  aid. 


Taken  together,  it  is  clear  that  the  design  of  feedback  plays  an  important  role  in  engendering 
appropriate  trust  in,  and  reliance  on,  combat  identification  aids.  These  findings  will  be  of  value  to 
both  designers  of  combat  ID  systems  and  human  factors  scientists  who  study  human  reliance  on 
imperfect  automation. 

Probabilistic  cue  integration 

Bryant,  D.  J,,  &  Smith,  D.  G.  (2009).  Impact  of  Uncertain  Cues  on  Combat 

Identification  Judgments.  Defence  R&D  Canada  -  Toronto  Technical  Report 

(Submitted). 

Bryant,  D.  J.  (2009).  Threat  Classification  Strategy:  Effect  of  a  Secondary  Task. 

Defence  R&D  Canada  -  Toronto  Technical  Report  (In  Progress). 

The  research  conducted  under  this  element  built  on  the  results  of  the  function  flow  and  decision 
analyses  performed  earlier.  Those  studies  described  the  “task  ecology”  of  CID,  laying  out  the 
decisions  to  be  made  by  soldiers  and  the  information  available  to  them.  In  this  work,  algorithms 
and  heuristics  for  the  decision  process  were  developed  and  compared  to  those  used  in  human 
decision  making.  By  identifying  human  models,  we  will  be  able  to  support  the  development  of 
procedures,  training,  and  technology  to  enhance  combat  ID  decision  making.  The  nature  of  the 
decision  making  process  determines  how  human  factors  affect  CID  and  can  also  help  us  better 
understand  the  kinds  of  physical  and  operational  conditions  that  will  challenge  soldiers. 

Bryant  and  Smith  [26]  report  the  results  of  an  experiment  that  examined  elements  of  human  CID 
decision  making.  In  particular,  the  experiment  provided  insight  into  the  way  people  deal  with 
uncertainty  associated  with  various  kinds  of  cues  (visual  and  behavioural)  that  indicate  whether  a 
target  is  friendly  or  hostile.  This  experiment  was  conducted  using  the  IMMERSIVE  platform,  in 
which  the  participant  assumed  the  role  of  a  dismounted  infantry  soldier.  Participants  completed 
blocks  of  trials  in  which  each  trial  comprised  a  human  figure  moving  into  view.  The  subject’s 
task  was  to  engage  (i.e.,  shoot)  only  those  figures  that  were  enemies.  Friendly  and  enemy  forces 
were  distinguishable  by  differences  in  uniforms,  equipment,  and  whether  they  are  identified  as 
friendly  in  the  combat  ID  system. 

The  initial  objective  of  the  experiment  was  to  investigate  the  impact  of  cue  uncertainty  on 
engagement  decision  making  (accuracy,  speed).  Two  factors  were  considered:  1)  the  type  of 
characteristic  that  is  uncertain  (visual  or  behavioural),  and  2)  the  salience  of  the  uncertain  feature 
(salient  or  non-salient).  These  factors  were  systematically  varied  across  blocks  of  trials. 
Uncertainty  as  to  which  characteristics  are  important,  or  diagnostic,  to  the  identity  of  targets  can 
impair  CID  [6].  This  is  especially  true  in  asymmetric  environments  in  which  the  enemy  uses 
diverse  equipment  and  attempts  to  blend  into  civilian  populations,  as  well  as  in  coalition 
operations  in  which  allies  may  use  different,  unfamiliar  equipment.  Determining  the  specific 
impact  of  uncertainty  of  different  types  of  cues  will  help  us  better  predict  the  impact  of 
uncertainty  in  operational  settings. 

The  results  indicated  that  both  hit  rate  (i.e.,  correctly  engaging  an  enemy)  and  false  alarm  rate 
(i.e.,  incorrectly  engaging  a  friend)  can  be  affected  by  uncertainty  associated  with  visual  and,  to  a 
lesser  extent,  behavioural  characteristics  of  targets  in  the  environment.  When  uncertainty  was 


associated  with  the  characteristics  of  friends,  the  false  alarm  rate  increased,  whereas  hit  rate  was 
primarily  affected  by  uncertainty  associated  with  potential  enemies.  In  both  cases,  effects 
depended  on  the  salience  of  the  characteristics  that  were  uncertain.  When  a  salient  visual  or 
behavioural  characteristic  of  friends  was  uncertain  (i.e.,  a  friend  may  possess  a  characteristic 
normally  associated  with  enemies),  subjects’  false  alarm  rates  tended  to  be  greater  than  that  of  the 
baseline  or  when  a  non-salient  characteristic  was  uncertain.  Thus,  subjects  were  more  likely  to 
make  a  serious  error  of  engaging  a  friend  in  these  cases.  When  the  characteristics 
diagnostic  of  enemies  were  uncertain,  participants’  hit  rates  declined  but  their  false  alarm  rates 
were  unaffected. 

In  another  experiment  [27],  we  contrasted  the  potential  use  of  different  decision  strategies  for 
CID  decision  making.  A  major  question  in  decision  research  concerns  the  value  of  heuristics  in 
relation  to  analytic  procedures.  Whereas  an  heuristic  is  a  simple  decision  procedure  that  offers 
the  potential  to  quickly  and  easily  solve  a  specific  problem,  an  analytic  procedure  promises  an 
optimal  solution  at  the  cost  of  extensive  computation  and  time.  Both  approaches  have  received 
empirical  support  and  both  can  be  successful  approaches  to  problem  solving,  and  this  precludes  a 
simple  conclusion  that  one  is  an  inherently  better  approach  than  the  other.  The  results  of  these 
experiments  have  indicated  that  people  can  employ  both  analytic  and  heuristic  decision  strategies 
but  an  individual’s  choice  of  one  or  the  other  appears  to  be  a  complex  interaction  of  individual 
and  situational  factors. 
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Lessons  learned 


There  are  three  major  outcomes  of  the  Applied  Research  Project  (ARP)  15au  (“Combat 
Identification”).  These  were:  a)  a  computer-based  methodology  for  studying  the  effects  of 
environmental  (e.g.,  types  of  visual  cues)  and  system  (e.g.,  blue-force  tracking)  factors  on  the 
speed  and  accuracy  of  human  combat  identification  judgments,  b)  an  information  accumulation 
model  of  human  combat  identification  decision  making,  and  c)  preliminary  experimentation  to 
validate  this  model  and  provide  insight  into  the  likely  effects  of  blue-force  tracking  and  rifle- 
mounted  combat  identification  assist  systems  on  human  decision  making.  These  outcomes  are 
documented  in  a  series  of  reports  published,  or  in  preparation,  by  DRDC  Toronto. 

The  following  are  the  major  lessons  learned  during  the  course  of  this  project. 

1.  It  is  possible  to  identify  decision  procedures  used  by  individuals  through  controlled 
experiments.  We  developed  a  methodology  to  examine  human  CID  decision  making  [26] 
[28]  which  can  be  used  to  contrast  decision  models. 

2.  A  relatively  simple  and  inexpensive  gaming  environment  simulator  can  be  used  to  study  CID 
decision  making.  The  IMMERSIVE  environment,  which  is  run  on  standard  personal 
computers,  is  low  cost,  portable,  and  offers  precise  control  of  experimental  factors. 

3.  Combat  identification  involves  three  main  components,  SA,  target  identification,  and  TTPs 
(which  comprise  knowledge  needed  to  interpret  information  pertaining  to  the  environment 
and  targets).  Failure  of  any  one  of  these  components  can  significantly  impair  CID 
performance.  Moreover,  failure  of  one  component  increases  the  difficulty  of  successfully 
performing  or  maintaining  the  others.  For  example,  insufficient  knowledge  of  TTPs  makes  it 
difficult  for  a  soldier  to  build  SA  and  to  correctly  interpret  cues  in  target  identification. 

4.  CID  activities  cover  four  stages/activities  of  any  combat  mission  [9]:  a)  Preparation,  b) 
Monitoring,  c)  Taking  action,  and  d)  Evaluating  the  outcomes  of  action.  Current 
technological  systems  proposed  to  support  CID  deal  only  with  the  second  and,  to  a  lesser 
extent,  third  of  these  stages.  Blue-force  tracking  systems  are  intended  to  improve  SA  and 
make  monitoring  easier  and  more  accurate.  Interrogate-Friend-Foe  (IFF)-type  systems  that 
warn  a  soldier  when  a  weapon  is  directed  at  a  friend  are  meant  to  assist  in  making  an 
identification.  These  systems  provide  no  support  for  preparation  (i.e.,  learning  about  the 
enemy,  terrain,  local  culture,  etc.)  even  though  our  function  flow  analysis  [9]  indicated  that 
this  knowledge  was  a  primary  contributor  to  successful  CID.  Fikewise,  there  is  no  support 
for  after-action  learning. 

5.  Asymmetric  conflict  increases  the  difficulty  of  all  components  of  CID.  Given  the  need  for  an 
asymmetric  opponent  to  remain  hidden  and  the  rapidity  with  which  that  opponent  can  change 
tactics,  it  is  difficult  for  our  forces  to  maintain  appropriate  TTPs.  There  is  more  to  learn 
(e.g.,  wider  range  of  potential  cues  to  identify  targets,  cultural  knowledge)  and  less  time  in 
which  to  learn  compared  to  traditional  state  versus  state  warfare.  SA  is  also  harder  to 
maintain.  The  enemy  can  conceal  itself  more  effectively  and  there  is  often  a  civilian  activity 
in  operational  areas.  Finally,  asymmetric  conflict  makes  TI  harder.  The  enemy  will  wear 
clothing  similar  to  that  of  civilians,  carry  a  range  of  equipment  that  may  be  unfamiliar, 


variable  from  individual  to  individual,  or  similar  to  that  carried  by  coalition  partners.  This 
decreases  the  usefulness  of  these  cues  for  making  unambiguous  identifications. 

6.  Soldiers  depend  on  a  range  of  visual  and  behavioral  cues  to  make  identification  judgments. 
SA  can  help  by  providing  expectations  that  speed  the  search  for  diagnostic  cues  and  aid  the 
interpretation  of  cues.  Poor  SA,  however,  can  also  hinder  identification.  An  inaccurate 
model  of  the  environment  can  slow  the  search  for  cues  and  guide  incorrect  interpretation  that 
can  lead  to  mis-identification. 

7.  We  do  not  know  exactly  how  soldiers  use  cues  in  making  identifications.  There  are 
numerous  models  of  information  aggregation  [11],  any  of  which  could  serve  as  a  valid 
procedure  for  CID  decision  making.  The  problem  of  determining  the  kinds  of  decision 
procedures  actually  employed  by  soldiers  is  simplified  somewhat  by  contrasting  broad  classes 
of  models,  such  as  formal/analytic  models  versus  intuitive/integrative  models.  Nevertheless, 
our  research  has  indicated  that  individuals  use  different  decision  strategies  for  a  given  task, 
depending  on  personal,  environmental,  and  educational  factors.  Not  knowing  how  a  soldier 
combines  cues  makes  it  harder  to  design  decision  support  systems  and  to  predict  how  a 
soldier  will  use  those  systems  [22], 

8.  The  effectiveness  of  a  decision  support  system  will  be  mediated  by  the  user’s  perception  of, 
and  trust  in,  the  outputs  of  the  system  [22],  Automated  systems  (e.g.,  BFT)  can  be  misused  if 
the  user’s  trust  is  not  well  calibrated  to  the  actual  accuracy  of  the  system.  Factors  that  affect 
a  user’s  trust  include  the  system’s  general  level  of  accuracy,  its  reliability  (i.e.,  propensity  to 
avoid  large  errors),  and  the  ability  of  the  user  to  accurately  gauge  the  accuracy  and  reliability 
of  the  system.  A  support  system  should  include  information  about  the  reliability  of  system 
outputs  to  which  users  can  refer  to  help  them  calibrate  their  trust  levels  [25]. 
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Project  manager’s  observations 


The  15au,  Combat  Identification,  project  was  largely  successful.  It  has  advanced  our 
understanding  of  human  CID  decision  making  as  well  as  our  understanding  of  how  human 
decision  makers  are  likely  to  interact  with  automated  CID  support  systems. 

Progress  in  this  project  has  been  made  largely  in  the  laboratory  setting,  albeit  with  significant 
results  being  obtained  through  interviews  with  SMEs.  The  positive  aspect  of  this  situation  is  that 
an  effective  computer-based  system  was  developed  to  study  CID  decision  making.  The 
IMMERSIVE  environment  proved  flexible  and  easy  to  use  while  offering  complete  experimental 
control.  This  simulation  environment  allows  measurement  of  a  wide  range  of  relevant  human 
performance  related  to  CID  decision  making.  It  is  also  readily  configurable  to  allow  study  of  CID 
decision  making  in  a  wide  range  of  different  environments.  One  downside  of  the  IMMERSIVE 
environment  is  its  low  fidelity  or  lack  of  realism.  A  computer  simulation  cannot  replicate  all 
aspects  of  the  CID  task  and  certainly  lacks  the  contextual  factors  of  an  operational  setting. 

Nevertheless,  the  IMMERSIVE  environment  should  remain  an  important  tool  because  human 
CID  decision  making  is  difficult  to  study  in  field  exercises.  The  Bold  Quest  (BQ)  exercises,  for 
example,  were  set  up  to  examine  the  technical  performance  of  various  BFT  systems. 
Unfortunately,  examination  of  human  factors  have  frankly  been  an  afterthought  in  the  design  of 
these  exercises.  Although  researchers  were  able  to  administer  surveys  to  participants,  this  cannot 
be  said  to  constitute  an  adequate  scientific  study.  To  perform  effective  studies  of  CID  decision 
making  in  a  field  setting  requires  (at  a  minimum): 

•  The  capability  to  directly  measure  human  decision  making  and  task  performance,  such  as 
decision  accuracy,  decision  making  latency,  and  information  use;  and 

•  Enough  control  of  the  design  of  an  exercise  to  allow  experimental  comparison  of  conditions 
based  on  important  theoretical  factors  (e.g.,  with  versus  without  the  use  of  a  particular 
support  system,  with  extensive  versus  impoverished  SA,  etc.). 

A  field  study  is  undoubtedly  an  expensive  and  time-consuming  endeavor,  but  we  cannot  simply 
“take  it  on  faith”  that  introducing  a  decision  support  system,  even  one  that  works  exactly  as 
designed,  will  necessarily  produce  better  human  performance.  When  such  systems  are  based  on 
the  intuitions  of  the  engineers  who  design  them,  incorrect  assumptions  about  human  cognition 
may  yield  counterproductive  results. 


Future  R&D  and  exploitation  activities 


This  section  outlines  future  research  directions  and  potential  exploitation  activities  for  CID 
research.  There  are  many  interesting  directions  for  new  research  associated  with  human  decision 
making  in  this  area.  However,  the  major  criteria  for  the  future  directions  outlined  in  this  section 
are  that  they  address  CF  needs  in  C4ISR  and  CID  and  are  also  consistent  with  the  vision  and 
goals  of  the  Human  Systems  Integration  Section  at  DRDC  Toronto. 


Research  priorities  for  the  CF  in  CID 

Achieving  good  SA  and  CID  are  challenging  given  the  increased  tempo  and  spatial  dispersion  of 
current  operations  and  the  likelihood  of  working  in  coalitions  with  nations  employing  different 
C2  systems  and  procedures.  Thus,  establishment  and  maintenance  of  a  CF-wide  SA  capability  is 
a  prime  goal  of  Command,  Control,  Communications,  Computers,  Intelligence,  Surveillance  and 
Reconnaissance  (C4ISR).  Likewise,  the  CF  continues  to  seek  improvements  in  C4ISR  and  CID 
that  will  directly  impact  SA  and  decision  making  in  network-enabled  operations,  performance  of 
distributed  teams,  intelligence  gathering,  information  operations,  and  enhance  the  development  of 
decision  support,  communications  systems,  doctrine,  and  training. 

A  key  priority  for  the  CF  is  the  investigation  of  CID  and  SA  technologies  that  have  the  potential 
to  prevent  fratricide.  One  such  technology  is  the  Battlefield  Target  Identification  Device  (BTID), 
which  is  a  millimetric-wave  (mmW)  CID  technology  developed  under  NATO  Standardization 
Agreement  4579  so  that  interoperability  can  be  achieved  in  a  Coalition  environment.  It  is 
currently  Canada’s  favoured  CID  solution. 

In  reality,  BTID  encompasses  a  family  of  CID  solutions,  all  based  on  the  above-mentioned  mmW 
technology.  The  BTID  Target  ID  interrogation  solution  was  demonstrated  and  assessed  at  the 
Urgent  Quest  operational  demonstration  in  2005.  The  BTID  digital  data  link,  a  Situational 
Awareness  /  Blue  Force  Tracking  (SA/BFT)  solution,  was  demonstrated  and  assessed  at  the  BQ 
Operational  Demonstration  in  2007.  A  proof  of  concept  of  the  BTID  Transponder  Airborne 
Platform  Surveillance  System  was  also  demonstrated  at  BQ.  The  CF  is  expected  to  continue  to 
pursue  these  technologies  in  the  future. 

New  technologies,  however,  are  only  part  of  any  solution  to  the  problem  of  fratricide.  The  C4ISR 
challenge  is  to  design  doctrine,  procedures,  and  training  that  will  allow  the  CF  to  optimize 
performance  and  the  effectiveness  of  tools.  For  this  reason,  human-oriented  research  will 
continue  to  be  an  essential  component  of  CID  research. 


Potential  directions  for  exploitation 

This  section  discusses  a  number  of  ways  in  which  the  results  of  the  current  project  could  be  used 
to  enhance  the  CF’s  CID  capabilities. 


Enhanced  TTPs 

The  lessons  learned  from  this  project  offer  suggestions  for  enhancing  doctrine,  tactics, 
procedures,  training  methods  for  individual  and  team  decision  making.  For  example,  it  seems 
that  the  preparation  phase  of  missions  is  currently  being  neglected  in  the  primarily  technology- 
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based  approaches  to  CID  (e.g.,  BTID).  These  technologies  provide  positional  information  on 
friendly  units  but  do  not  help  soldiers  learn  the  characteristics  that  distinguish  enemy  and  neutral 
entities  from  friends,  nor  do  they  provide  contextual  information  to  help  interpret  visual  data. 


Refinement  of  CID  decision  models 

This  project  has  identified  a  large  number  of  formal  decision  making  models  applicable  to  CID 
(see  [11]).  Moreover,  results  of  several  experiments  [27]  [28]  indicate  that  different  individuals 
employ  different  decision  processes  within  the  same  task  environment.  Thus,  rather  that  seeking 
a  single  universal  model  of  CID  decision  making,  it  is  likely  necessary  to  create  an  integrative 
framework  in  which  to  understand  how  soldiers  will  make  CID  decisions. 

The  AEAT  model  [12],  which  is  a  generalized  version  of  an  information  aggregation  model, 
could  serve  as  a  preliminary  framework  in  which  to  explore  how  soldiers  search  for  and  aggregate 
discrete  cues  in  the  CID  process.  This  model  combines  several  key  psychological  processes,  such 
as  psychological  arousal  or  readiness  and  threshold  response.  This  framework  can  be  augmented 
in  the  future  by  integrating  more  specific  models  of  how  information  is  aggregated  and  how 
information  translates  into  neural  arousal  in  the  decision  process. 

Another  model  is  the  Integrative  Combat  Identification  Entity  Relationship  (INCIDER),  which 
was  developed  in  the  United  Kingdom  to  model  the  decision  making  process  culminating  in 
outcomes  of  combat  ID  encounters.  It  considers  SA,  the  contribution  of  sensors  (technologies), 
and  personal  characteristics  (e.g.,  personality,  experience,  expectations)  as  an  inter-related 
variable  set  that  enables  predictions  of  the  outcome  of  an  identification  process  undertaken  by  a 
single  decision  maker  observing  a  single  unknown  entity.  The  INCIDER  model  could  also  be 
translated  as  a  framework  to  deal  with  specific  CF  issues  pertaining  to  CID. 


Development  and  assessment  of  decision  support  systems 

One  goal  in  developing  a  cognitive  model  of  human  CID  decision  making  has  been  to  provide  the 
basis  for  making  intelligent  choices  concerning  the  design  of  decision  support  for  CID.  We  can 
further  advance  this  goal  by  modeling  key  aspects  related  to  SA,  decision  criteria,  and 
categorization  learning,  including  the  interaction  of  SA  and  CID.  Future  decision  support 
development  should  take  into  account,  at  the  earliest  stages  of  concept  development,  the  basic 
elements  of  human  decision  making  revealed  by  this  project;  i.e.: 

•  Humans  aggregate  perceived  cues,  weighted  by  their  perceived  diagnosticity; 

•  Cue  perception  is  affected  by  a  range  of  individual  and  situational  factors  that  can  result  in 
significant  biases  in  interpretation; 

•  Different  decision  makers  use  different  information  aggregation  procedures  based  on 
radically  different  principles  (e.g.,  analytic  versus  intuitive/heuristic  processes)  such  that  the 
outputs  of  a  given  decision  support  tool  can  be  interpreted  in  very  different  ways; 

•  The  user  of  a  decision  support  tool  will  respond  to  the  perceived  reliability  of  that  tool  and 
could  potentially  over-  or  under-utilize  a  tool  if  its  reliability  has  been  misperceived;  and 

•  HMI  design  is  critical  to  engendering  appropriate  trust  and  reliance  on  the  automated 
decision  aid;  presenting  a  more  integrated  display  format  yields  more  appropriate  reliance 
on  a  system. 


A  cognitive  model  can  serve  as  the  basis  for  developing  a  methodology  to  assess  the  impact  of 
soldier  assist  devices  on  CID  performance,  as  well  as  performance  of  other  tasks  carried  out  while 
doing  CID.  Thus,  any  assessment  should  consider  the  impact  of  decision  support  for  CID  on 
cognitive  workload  and  a  range  task  performance. 

A  decision  support  system  should  help  the  human  arrive  at  better  decisions.  The  usual  design 
approach  to  developing  decision  aids,  however,  can  create  unanticipated  problems  [21], 
Computer  advice  invokes  unfamiliar  deliberation  and  ambiguity  about  its  interpretation  within  the 
context  of  a  particular  decision  and  about  system  performance,  for  which  neither  decision  makers 
nor  system  developers  are  prepared.  The  results  of  this  project  provide  a  first  step  in 
defining  the  mechanisms  by  which  a  computer’s  decision  becomes  helpful  to  a  human 
decision  maker,  identifying  common  fallacies  in  the  decision-aid  concept,  setting  minimum 
standards  that  decision  aids  must  meet  if  they  are  to  be  useful,  pointing  to  the  crucial  role  of  trust, 
and  illustrating  how  system  developers  can  integrate  expert  systems  following  rational  principles 
of  trust  [22], 


Areas  of  future  research  interest 

This  section  describes  research  topics  that  could  be  pursued  in  the  future.  These  areas  address  the 
ongoing  need  to  enhance  the  CF’s  capability  to  rapidly  and  accurately  identify  friendly,  enemy 
and  neutral  forces,  manage  and  control  the  battlespace,  optimally  employ  weapons  and  forces, 
and  minimize  the  risk  of  fratricide. 


Assessment  of  technologies 

In  considering  the  human  dimension  of  decision  support  systems,  there  are  two  important  issues 
concerning  any  decision  support  technology  to  be  implemented: 

1.  Functionality/System  Performance:  Do  the  combat  identification  and  situation  awareness 
technologies  under  investigation  individually  and  collectively  enhance  the  capability  of  the 
pilots  to  identify  friends,  foes,  and  neutrals  in  the  battlespace? 

2.  Warfighting/Operational  Impact:  Do  the  systems  individually  and  collectively  make  a 
positive  contribution  to  fratricide  reduction  and  combat  effectiveness? 

Currently,  in  exercises  such  as  Ex  BQ+,  the  emphasis  has  been  on  evaluating  the  functionality  of 
BFT  technologies,  with  less  attention  paid  to  the  question  of  how  they  affect  human  decision 
making.  This  situation  creates  an  opportunity  to  consider  approaches  to  evaluating  the  impact  of 
CID  decision  support  systems.  Research  that  investigates  the  effectiveness  of  decision  support 
technologies  in  enhancing  the  overall  CID  performance  of  individuals  and  teams  should 
accomplish  the  following: 

•  Evaluate  whether  the  technologies  are  being  used  by  soldiers  as  designed; 

•  Determine  whether  the  technologies,  when  used  properly,  have  the  intended  enhancing 
effect  on  human  performance; 

•  Determine  whether  the  technologies,  when  used  properly,  do  not  have  unintended  negative 
impact  on  the  performance  of  other  operationally  necessary  tasks;  and 

•  Provide  a  high-level  assessment  of  the  appropriateness  and  effectiveness  of  the  technology 
training  (predominantly  from  users’  perspective). 
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The  proposed  work  could  support  an  effort  to  develop  standardized  metrics  for  the  evaluation  of 
decision  support  systems.  This  would  allow  the  measurement  and  comparison  of  CID  and  SA 
technologies  on  the  basis  of  timeliness,  accuracy,  and  completeness.  A  standardized  evaluation 
methodology  would  be  valuable  for  addressing  numerous  research  questions: 

•  How  will  the  introduction  of  new  technologies  impact  upon  operational  tempo  (measured  by 
the  reduction  of  decision  cycles  and  increase  in  engagement  ranges)? 

•  How  will  the  introduction  of  new  technologies  impact  upon  fratricide  rates?  (Will 
warfighters  trade  off  safety  for  improved  effectiveness?) 

•  What  impact  does  Joint,  Coalition  and  CID  training  have  on  operational  effectiveness? 

•  How  much  information  can  the  human  take  in  and  comprehend  in  high  pressure  combat 
situations? 

•  How  effective  are  CID  training  systems? 

•  When  in  the  decision  cycle  should  SA  systems  be  used? 

Training  and  development  issues 

Gaining  operational  experience  takes  time,  during  which  soldiers  are  at  heightened  risk.  The  CF 
needs  an  effective  and  economical  way  to  provide  CID  training  prior  to  deployment  to  operations. 
Training  should  also  be  able  to  flexibly  adapt  to  changes  in  operational  setting  or  requirements. 

A  worthwhile  avenue  of  research  to  pursue  is  the  development  of  a  computer-based  CID  training 
system  that  enhances  target  identification  skills  and  helps  soldiers  build  a  knowledge  base  to 
improve  SA  when  deployed.  Specifically,  the  potential  of  the  IMMERSIVE  environment  as  a 
training  instrument  should  be  evaluated. 

A  computer  CID  training  instrument  could  improve  both  the  rate  of  learning  of  soldiers  as  well  as 
the  ultimate  level  of  performance.  Specifically,  soldiers  could  acquire  high  levels  of  expertise 
and  operationally-relevant  knowledge  prior  to  deployment,  leading  to  significantly  increased 
levels  of  combat  effectiveness  and  decreasing  the  risk  of  fratricide  and  neutricide.  Computer- 
based  training  has  the  potential  to  reduce  training  costs  and  manpower  requirements. 

The  genera  1  effort  to  improve  training  options  could  be  broken  into  several  specific  areas 
of  research: 

1.  Case-based  versus  rule-based  learning:  This  area  focuses  on  model  development  and 
evaluation.  There  are  two  broad  approaches  to  CID  learning  that  could  be  contrasted.  The 
first  is  an  experiential,  case-based  view  in  which  a  soldier  learns  to  use  target  and  situation 
cues  that  are  diagnostic  by  building  memory  representations  of  instances  in  which  hostile 
targets  are  encountered.  The  other  is  a  rule-based  view  in  which  a  soldier  extracts  underlying 
rules  or  regularities  of  the  operational  environment  and  learns  to  use  these  to  predict  potential 
hostile  entities.  These  views  have  implications  for  the  types  of  training  that  would  be  most 
effective  and  so  it  is  important  to  distinguish  which  offers  the  better  description  of  actual 
soldier  performance. 

2.  Effects  of  CID  systems  on  soldier  performance:  CID  decision  support  systems  will  be 
deployed  in  the  near  future.  An  unexplored  issue  has  been  what,  if  any,  effect  do  these 


systems  have  on  training  and  ultimate  performance  of  soldiers?  Will  soldiers  who  have 
trained  to  use  support  systems  come  to  rely  too  heavily  on  them,  possibly  failing  to  learn  how 
to  distinguish  targets  based  on  their  own  observation? 

3.  Requirements  for  a  training  system  interface:  If  a  computer-based  training  system  can  be 
developed,  research  will  be  needed  to  examine  the  design  of  an  optimal  interface  for  the  CID 
training  system.  One  issue  to  be  addressed  is  the  degree  of  realism  needed  to  promote 
learning  that  is  transferable  to  operational  contexts.  In  addition,  interface  design  issues 
concerning  CID  systems  (blue  force  tracking,  IFF)  can  be  explored. 
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